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I . INTRODUCTION 

The reliability of a system composed of N statistically independent 
components, is defined: 



R(T^) = Probability (T ^ T^) 



where T denotes the time of first system failure. A significant practical 
problem is that it is often reasonably easy, and comparatively inexpensive, 
to determine information about the reliability of separate components, 
denoted i=l,2, ..., N , but fairly difficult and extremely 

expensive to determine total system reliability directly, since failure 
testing often destroys the system. 

In the most common analytically treated case each of the component 
types has an assumed exponential failure rate, i.e., 

-XT 

R^(To) = e ^ ° , i=l,2 N , (1) 



the system reliability is given by 



-XT 

R(T ) = e ° 
o 




( 2 ) 



and a statistically exact procedure for obtaining an upper bound on X 
from data on component failures, called the LR procedure, has been develop- 
ed by Lieberman and Ross [1]. In this procedure items of each type 

component, with individual observed failure times i“l>2, ..., N and 

j“l,2, .••, are tested. (We shall assume without loss of generality 

that kj ^ , j=l,2, ..., N-1.) A time U is defined: 



1 



(3) 



U =“ 





i.e., U is the cumulative time at which one first exhausts all the com- 
ponents of one type. The number of each type of component which has been 
used up to this time is given by: 



Then, if 



j 

"l ■ J 1 I ^ 1 Vn>- 

«.=! 



N 

i=l 



(4) 



(5) 



(i.e., total components used), Lieberman and Ross showed that 2XU follows 
2 

the distribution; hence upper bounds for X can be estimated. 

A major drawback of the LR technique is that in using K as defined 
by (4)-(5), one discards the "information" known about the 

N 

NL {k^-n^} (6) 

i=l 



components which do not fail by the time U. 

An immediate consequence of (4) is that the LR procedure is Data- 
Order Dependent . By this we mean that permuting the second subscript on 
(i.e., in essence permuting the order in which the failures are 
observed) can alter n^ , and hence the estimated bound. For example, 
consider the following hypothetical observed inter-failure times in two 
tests: 



2 



Test A 



Test B 



Component 


n: 


T 

11 


Component 


in: 


T 

21 


Component 


ifl: 


T 

11 


Component 


#2: 


T 

21 



0-75 , T^2 “ 0-25 
0-80 , T^2 “ 1.25 

0.25 , T^2 = 0-75 
1.25 , T 22 = 0.80 



Observe the data are identical except for their order (i.e., which values 

are associated with which second subscripts), and U = 1.00 in both tests, 

however K = 3 in Test A but only K = 2 in Test B. This leads to upper 

bounds (at the 95% confidence level) for X of 6.30 and 4.75 respectively. 

This dependence on the order in which the second subscript of the T^^ are 

assigned causes a potentially wide variance in the bounds for X for the 

identical set of individual component failure data. This can lead to 

significant practical problems, for often interfailure data on individual 

components is presented with no reference as to the order in which the 

failures occurred. Thus there is no preferred ordering, and the analyst 

is faced with the formidable task of deciding on the ”best** way to assign 

the T , . . 
ij 

We shall investigate, using analytic and simulation techniques, the 
impact of this ordering, and other parameters, on the mean and variance 
of the bounds produced for X . 
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II. ANALYSIS OF THE LR METHOD - TWO COMPONENT CASE 

Each application of the LR procedure to a given set of data involves 
a simultaneous observation of two random variables - the discrete random 
variable K , and the continuous random variable U. Given the confidence 
level, a , desired, then the estimated bound for A , 



2U 



is Itself a random variable, formed as the ratio of two random variables, 

2 

since the value can be considered a discrete random variable with 

the same number of realizable values as K , and, except for different 

sample values, the same probability law. (It is easily seen that for a 

given set of failure times, , the effect of permutation of the data 

is to alter the value of K , not U .) Thus, we start by considering the 

properties of the random variable X, when the system consists of two 

b 

components . 

A key step in Lleberman and Ross' paper is their defining a binomial 
random variable, , where 

1 1 with probability X^/ 

2 with probability X^/ (^^^^2^ * 

and showing that has the same probability law as the expected failure 

pattern of the type 1 and 2 components. Thus, they show the unconditional 
probability density function for K is: 
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P{K=k} = -< 






/k-1 \/A_\ ’'i /A_\ 

/k-l \ / \^~^2 / ^2 \^2 

^k2-lj \x^+X2/ ' 



< k2 - 1 , 



(7) 






(Note there is an obvious misprint in this formula in the original paper.) 
The LR procedure arises from this observation, plus the well-knovm result 
that U has the conditional density function 



/ N k-l (^i+X2)u 

“ (k-l)l ^ ® 



( 8 ) 



But, it follows immediately that has the conditional density function: 



,(X J = f. 



’x^ |K=k'"e' ^U|K=k I 



'X2K<“>\ X2^») 



2X' 



1 1 



X (k-l) I 
e 



2 , s1 k 



XX2k<») 



2X 



e J 



exp -Xx2i^(a)/2X^ , X = X^ + X^, (9) 



and thus the conditional mean: 



E(X^|K=k) 



00 

■I. 



X f> lv-,,(X )<JX 



■x^lK-: 



k e e 






hn J 



XX2k^°‘) 

2 (k-l) I 



CO 

/ 



k-2 -T, 

T e dx , 
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where t = 



2A 



Observe then: 



E(X^|K=k) 



AX2k(“) 

2(k-l) 



k > 2 



k = 1 



( 10 ) 



Since the (unconditional) mean value of the upper bounds is: 



E(A^^) = E(A^|K=k)P(k-k) , 

k=k- 



( 11 ) 



it follows that, unless k. > 2 , E(A ) = «> . In fact, the following 

-L b 

proposition is an immediate consequence. 

th 

Proposition: For — ^2 * ^1 — moment of A^ (the Lieberman 

and Ross upper bound to A) will be unbounded. 

Note also from (10) and (11) , that when ^ 2 , 

E(A,.) ^ P(K=k)i = A E([x2k(«)/2(k-l)]). (12) 

k=k- '' 



It may be observed from the expression for P(K=k) in equation (7) that 

E(X ) depends, not on X , but on X /X« , since P(K=k) depends on this 
b i / 

ratio* Thus we have shown: 

Theorem: Given k., , k« and X^/Xr^ , the mean estimate for X^ 

^ 1 Z 1 Z D 

produced by the Lieberman and Ross technique will be the same constant 

percentage of X , independent of the value of X , 
th 

The r conditional moment of X^ is given by 

r 



E(A^|K=k) = A^ 






(k-,r-l).! 1 < r < k 

(k-1) ! » ^ ^ • 



(13) 
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In particular, 



2 

a 




X 



2 




(14) 



Figure 1 shows {E(A^ |K=k)/X} and {a^ /X} for a representative 



range of values of k • Note that for k > 10 the variation in the mean 
value is quite small. The above results easily generalize to the case of 
more than two component types. 

As noted in the introduction, the "lost** data in the LR method is 
represented by those components which have not failed by U . (See 
equation (6)). Except in the case of simultaneous failure of all component 
types at U , there will be at least one component left of each type except 
one. Also, the LR method requires use of 100% of the data for one component 
type, hence the number of this type of component should not be included in 
any measure of relative data loss. Based on this discussion, we shall 
define the percentage data loss in any application of the LR method as 



where f is the index of the component type which was 100% exhausted, 
i.e., ^ ^ defined by (6). 




(13) 



i^f 




N 



±¥f 
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In the two component case, the mean of PL can be computed fairly 
straightforwardly from (7). Observe that the unconditional probability 
that the first type of component terminates the test is: 



kj^+k2~l 



P(f=l) 



E 

k=k. 



'k-1 
k,-lMX,+A 



vk-k. 



1 2> 






1 

(k^-1) 1 







. (14) 



But the conditional density function for n 2 , given f=l , is: 






(15) 



Thus the conditional expectation on n 2 is 



k2-l 



k+k-“l\ 



E(n |f-l) " V k( ^ 

-feo, Vr" 



^1 \^l/ X “1 

- ' 1/-^' [P(f=D] ( 16 ) 



,x^+x^ 






A similar expression is derivable for E(nj^|f=2). Thus, based on (13), 
we see that the expected data loss for the two component case is: 



E(PL) 



k^-E(n^|f«2) 



P(f=2) + 



;k2-E(n2|f=l) 

I kZ 



P(f“l) 



- 1 - ^ E(nj^|f=2)P(f=2) - 




E(n2|f=l)P(f=l) 



8 



1 





(17) 



(Note (17) is a function solely of k^, and 

Figure 2 shows curves for several combinations of k , and k^ • Observe 
that each curve appears to have a unique minimum, occuring at or near the 
point 






= 1 



(18) 



i.e., when both subsystems have the same expected life before exhaustion. 
Therefore we also plotted the percentage of lost data as a function of 



and this is displayed in Figure 3. We would conjecture that the minimum 
data loss occurs when (18) is satisfied, and the curves indicate a 
monotonicity in terms of total number of components, but we cannot show 
this conjecture analytically. 

Expanding (17) to cover n > 2 , while possible, is computationally 
less useful since multiple series become involved. 
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III. DESCRIPTION OF THE SIMULATION FOR LR TEST 



As part of this study, a simulation model was developed to generate 
individual component failure data, and perform the statistical analyses of 
the basis LR procedure. The programs were all written in FORTRAN IV, and 
run on the IBM 360/67 at NPS, using certain subroutines from both the NPS 
and IMSL libraries. 

The simulation was developed to consider up to twenty different 
individual components of each of six different types. Individual components 
of the same type are assumed to be i.i.d, and the serial failure of all 
components of a type terminates the test. The simulation initially 
generates 120 shuffled random numbers, distributed exponentially with X=l. 
The subroutine package LLRANDOM, as described in [2], is used. These 
numbers are then divided into blocks of twenty, and each block adjusted to 
produce exponentially distributed failures with (i«l,2, ..., 6) by 

dividing by . Then, for each component type, cumulative failure data 
is constructed from the individual component failure times. From this 
data, the first component type to be exhausted is determined, and the 
number of components of each of the other t)rpes that have failed up to that 
time, is also determined. With this data, the quantities U and 2K 
needed for the LR test are determined, and the estimated bound for X is 
computed using the subroutine PRCHI from the IMSL package to compute the 
inverse chi-square table. 

At this time, the routine also collects statistics on the unused 
data. Specifically, it computes the percentage of usable data lost, defined 
by (13). Repeated iterations (normally 300) of the simulation are run for 
each choise of k^ and X^ . At the completion of these iterations. 
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additional statistics are compiled on the mean, variance and 100a% level 
of the estimates on reliability obtained from the individual trials. 

Initial testing and validation of the simulation was carried out for 
the two component (N=2) case. Runs were made with various combinations of 
values for X^, and (Table 1). Each run consisted of 300 

iterations of the LR procedure, where each iteration consisted of generating 
one set of corresponding LR estimate for X . Each set of 

pseudo-random failure times was used only once, and without any reordering. 
Validation consisted of comparing the results of several simulations to 
predicted values. First was the comparison of the 100(l-a)% estimate of 
reliability to the true reliability. Since the LR procedure is an exact 
bound for X , and since, in these tests, we used an a=».95 confidence 
level, we expected that on each run, 95% of the bounds would fall above the 
true lambda. Thus, after each iteration, we ordered the bounds on X in 
increasing order, and chose the 15th (out of 300). The ratio of this 
value to the true reliability, X^ = (X^ 4* X^) was computed, and compared 
to unity. Figure 4 shows, as a function of X^ , the scatter diagram of 
ratios obtained for runs. 

The second comparison made was between the mean value of the X^ that 
were computed in the simulation and the theoretical mean predicted by 
equation (12). The results of this comparison are shown in Figure 5, where 
the X^ (the actual X used to generate the data) is plotted on the 
horizontal axis, and the vertical axis displays the ratio: 

where: X is the (arithmetic) mean of the computed upper bounds and 

b 
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E(X, ) Is the theoretical mean, (12). 

D 

The final comparison made was between the average actual data loss in 
a run, and the expected data loss predicted by (17). The result of this 
comparison are shown in Figure 6, where is on the horizontal axis and 

(PL/E(PL)) Is on the vertical. Here PL denotes the (arithmetic) mean 
percentage of data lost in the iterations of a run, and E(PL) is as given 
in (17). 

As can be seen from Figures 4-6, the simulated values were all in 
acceptable agreement with the predicted ones, and therefore, we concluded 
the simulation Itself is valid. 
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IV. THE EFFECT OF LOST DATA ON THE LR METHOD 



With the simulation, as described in Section III debugged and validated, 
we proceeded to investigate the effect of lost data on the LR method. This 
investigation proceeded along two major lines. First was consideration of 
the effect when no prior ordering of the data was used, i.e., the basic LR 
method. In this instance we were particularly interested in any relation- 
ship that might exist between the percentage of data lost and the accuracy 

of the bound, X , derived. Our second consideration was the effect on 
b 

the estimated bounds of a priori ordering of the data, e.g., ordering in 
order of increasing inter-failure times. As noted in the introduction, 
such methods will produce biased estimates for the bounds, and a major 
concern is whether the variance of the bounds is smaller than that of the 
standard LR method, and whether the bias is predictable. 

To investigate the relation between the lost data and the accuracy of 
the LR bound, the simulation was programmed to produce, at the end of each 
run, a scatter diagram showing the relative accuracy of the estimate 
(A^/At) versus the percentage of lost data on each iterations. The number 
of iterations was increased so that each run consisted of 500 iterations. 
Several representative such diagrams are shown at Figures 7-12. Although 
each case evidenced some “tightening'* of the group of estimates as the 
percentage of lost data decreased, the magnitude of this effect is really 
noticeable only when the total sample is small. This, of course, relates 
to the fact that the most rapid changes in Figure 1 occur for K £ 7 . It 
is also worth noting that the amount of scatter tended to decrease for 
— 1.0 . Again this seems intuitively clear. 
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To study the effect of ordering, the program was modified so that 
after the individual component interfailure times were generated, they were 
rearranged into increasing order. This clearly will produce a biased high 
upper bound, since it, in effect, includes only the least reliable compon- 
ents in satisfying: 

"l 

k»l 

Figures 13-15 display the consolidated results of 370 runs of 300 
iterations each, with a run corresponding to different values of k^, 

and X^ . Note that in each of these graphs the horizontal variable is 
chosen as 



L 



12 



(^ 1*^2 



’ ^ 1^2 I ’ 



(19) 



rather than = (X^ + X^) • The reason for this was that, after some 
reflection, we felt that the reordering described here should have decreas- 
ing effect as L^2 deviated from unity. (Note that L^2 simply reflects 
the ratio of expected failure times for all the components of each 
individual type.) The general trend in these figures is consistent with 
our expectations, i.e., the bounds for X (denoted X^) are consistently 
biased higher than their counterparts in the unordered case (Figures 13-14) , 
and the average data loss is consistently less than in the unordered case. 
However, on close inspection, observe that the dispersion in Figure 13 
seems more marked than that in the unordered case (Figure 4) , and more 
pronounced near This we had not expected, and therefore was in- 

vestigated in some more detail. 
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After some consideration, we decided that the accentuated spreading 



(especially near arose primarily from the tendency of the expo- 

nential distribution to produce some highly unreliable components (i.e., 
0), which could produce a marked variation in estimates for tests 
with small numbers of components. To confirm that this apparent increased 
spreading is actual, in Figures 16-17 we compared dispersions of the 
estimates in the ordered scheme with those in the unordered scheme. Spec- 
ifically, for each iteration of each run, bounds for were produced, 

both using prior ordering (denoted by X° ) , and without prior ordering 

bi 

(denoted X^ . ) . Then, for each run the ratios of actual magnitudes of 
bi 

variances observed: 



" - e2(x“^)} ’ 

and the ratio of relative variances: 



( 20 ) 



R 



r 




were computed, where the usual estimates are used, e.g.. 



( 21 ) 



= ’^bi 



1 

300 



300 

E 

i=l 



bi 



( 22 ) 



Finally, as a function of L ^2 > plotted (Figure 16) and 

(Figure 17). As expected, for all intents, always exceed unity. 

That is, the actual variance of the ordered scheme exceeded that of the 
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unordered scheme. Furthermore, the relative variance of the ordered scheme 
exceeded even the relative variance of the unordered scheme a significant 
portion of the time when > 0.5 . As we noted, for small L ^2 

expected effect of ordering should be less noticeable, and we attribute the 
fact that the relative variance here, as given by (21), did not exceed 
unity primarily to the fact that ^ ^b * 

Figures 13-17 do not seem to indicate that a priori arrangement of 
the component failure data in order increasing interfailure time offers 
any significant improvement over random ordering, largely because the 
variances displayed in Figure 14 suggest that prediction of the bias in- 
troduced by the ordering is not predictable. (Actually, this comment must 
be strongly qualified, since in Figure 14 we have expressed the variance 
based on the single parameter, L ^2 • is still quite possible that a 

more predictable relation could emerge were we to retain k 2 and 

(X 1 /X 2 ) as three independent parameters.) 

We also feel that any other a priori ordering algorithm will be no 
more successful in producing lower variance bounds as long as one attempts 
to predict the bias using a sing le parameter. There are, however, two 
other possibilities that should be investigated. One, as noted above, is 
to include several parameters in the prediction. The second is to 
recognize that every different ordering of the data produces an estimated 
bound, and each is, statistically, equally valid. This observation means 
that each LR test produces data which yield several samples from the same 
population. The implication is then that one should consider all the 
possible bounds that can be estimated from all the different possible 
orderings of the data. Thus, for example, the data in section I should be 
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viewed as yielding that both 6.30 and 4.75 are 95% confidence (upper) 
bounds on X , and this is certainly stronger than the statement that 
either one alone is such a bound. (Note it might be argued that this will 
produce an unmanageably large number of samples when the number of failed 
components is large. However, as reference to Figure 1 indicates, where 
the number of failed components is large, the variance on all the bounds 
is small, and so some randomly chosen smaller sample could be used.) 



17 



V, 



SUMMARY AND CONCLUSIONS 



Given a set of component interfailure times to be converted into an 
estimate on system reliability by the Lieberman-Ross technique, it is well- 
known that one parameter ("U") needed for the procedure is uniquely deter- 
mined, Irrespective of the order of the data, while the value of the second 
parameter ("K") is data order dependent and not uniquely determined. 
Furthermore, the design of the LR procedure requires that, in general, 
some relevant, valid data be discarded. The purpose of this paper was to 
consider the effects of this "lost" data and the data order. 

Since the data order determines the degrees of freedom for the chi- 
square test, we considered the distribution of estimated bounds on X 
produced for a fixed degree of freedom. Formulas were presented for the 
mean and variance of these bounds, and these were shown to be constant 
percentages of X for fixed K . We also programmed a simulation to 
investigate the impact of a priori ordering of the failure data. The 
ordering we chose was in order of increasing interfailure time (least 
reliable first), which is equivalent to including maximum data. The results 
of this simulation showed that this a priori ordering not only produced 
biased bounds for X , but, expressed in terms of the single parameter 

, these estimated bounds were more dispersed than the estimates 
produced without prior ordering. Thus we concluded it appears that a 
priori ordering offers no improvement over the basis LR procedure, as long 
as one attempts to predict the resultant bias in terms of a single 
parameter. In future investigations we hope to consider both multi" 
parameter prediction of the prior ordering bias, and improvement in the 
estimated bounds possible by utilizing several reorderings of the data. 
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TABLE 1 



VALUES OF 



2 


2 


0.001, 


0.005 


2 


3 


0.001, 


0.005 


2 


5 


0.001, 


0.005 


3 


3 


0.001, 

0.050 


0.005 


3 


5 


0.020, 


0.050 


5 


5 


0.001, 


0.005 






0.050, 


0.100 


5 


10 


0.050, 


0.100 


10 


10 


0.001, 


0.005 






0.050, 


0.100 



RUNS AT EACH X 

t 

20 

0 . 010 , 0.020 10 

0 . 010 , 0.020 10 

0 . 010 , 0 . 020 , 10 

0.100 10 

0.010, 0.020, 10 

0.200 

0.200 10 

0.010, 0.020, 10 

0.200 



Each run consisted of 300 iterations of an LR failure test using fixed, 
but randomly chosen values of X^, X 2 , subject to ^2 °° ^t* 
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Dependence of Mean and Variance of Bound 
on Data Size 



Figure 1 
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Figure 2 
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Figure A 
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Test of Simulation for Predicted Mean Bound 
Figure 5 
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Test of Simulation for Predicted Lost Data 
Figure 6 
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